NPS ARCHIVE 

ru’i^foWn. postgraduate school 

Monterey, California 




THESIS 



AN ANALYSIS OF THE HR AND FIR 
WIENER FILTERS WITH APPLICATIONS 
TO UNDERWATER. ACOUSTICS 

by 

Natanael Ruiz Fontes 
June 1997 

Thesis Advisor: Charles W. Therrien 

Second Reader; Anthony A. Atchley 

Approved for public relesise; Distribution is unlimited. 



Thesis 

R8583 



' ox LIBRARY 

■^G^ADUATIE SCHOOL 



DUDLEY KNOX LIBRARY 
NAVAL POSTGRADUATE SCHOOL 
MONTEREY, CA 93943-5101 



REPORT DOCUMENTATION PAGE 


Form Approved OMB No. 0704-0188 


Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instruction, 
searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send 
comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, 
to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, 

Arlington, Va 22202-4302, and to the Office of Management and Budget, Paperwork Reduction Project (0704-0188) Washington DC 20503. 


1. AGENCY USE ONLY (Leave blank) 


2. REPORT DATE 
June 1997 


3. REPORT TYPE AND DATES COVERED 
Master’s Thesis 


4. TITLE AND SUBTITLE AN ANALYSIS OF THE HR AND FIR WIENER 
FILTERS WITH APPLICATIONS TO UNDERWATER ACOUSTICS 


5. FUNDING NUMBERS 


6. AUTHORS Ruiz, Natanael F. 


7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 
Naval Postgraduate School 
Monterey CA 93943-5000 


8. PERFORMING 
ORGANIZATION 
REPORT NUMBER 


9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 


10. SPONSORING /MONITORING 
AGENCY REPORT NUMBER 


11. SUPPLEMENTARY NOTES The views expressed in this thesis are those of tl 
the official policy or position of the Department of Defense or the U.S. Go 


le author and do not reflect 
vemment. 


12a. DISTRIBUTION/AVAIL ABILITY STATEMENT 

Approved for public release; distribution is unlimited. 


12b. DISTRIBUTION CODE 


13. ABSTRACT( maximum 200 words) 

A detailed analysis of the performance the Wiener optimal filter for estimating a signal in additive noise is 
carried out. A first order AR model is assumed for both the signal and noise. Both IIR and FIR forms of the 
filter are considered and expressions are derived for the processing gain, mean-square error and signal distortion. 
These measures are plotted as a function of the model parameters. 

This analysis motivates a generalized form of the Wiener filter, which can improve the signal distortion. An 
analysis of this more general filter is then carried out. 

A practical noise removal algorithm based on short-time filtering using the generalized filter is also described, 
and results of applying the algorithm to some typical underwater acoustic data are presented. 


14. SUBJECT TERMS 

Wiener Filter, IIR, FIR, Applications to Underwater Acoustics. 


15. NUMBER OF 
PAGES 92 


16. PRICE CODE 


17. SECURITY CLASSIFI- 
CATION OF REPORT 

Unclassified 


18. SECURITY CLASSIFI- 
CATION OF THIS PAGE 

Unclassified 


19. SECURITY CLASSIFI- 
CATION OF ABSTRACT 

Unclassified 


20. LIMITATION 
OF ABSTRACT 

UL 



NSN 7540-01-280-5500 Standard Form 298 (Rev. 2-89) 



Prescribed by ANSI Std. 239-18 298-102 



1 




m 





11 



Approved for public release; distribution is unlimited. 



AN ANALYSIS OF THE HR AND FIR WIENER FILTERS 
WITH APPLICATIONS TO UNDERWATER ACOUSTICS 

Natanael Ruiz Fontes 
Lieutenant Comma^nder, Brazilian Navy 
B.S.j University of Sao Paulo, 1988 

Submitted in partial fulfillment of the 
requirements for the degree of 

MASTER OF SCIENCE IN ENGINEERING ACOUSTICS 

and 

MASTER OF SCIENCE IN ELECTRICAL ENGINEERING 

from the 

NAVAL POSTGRADUATE SCHOOL 
June 1997 



/ Y-y/^ 
i-/A 



A'M-. ot 



DUDLEY KNOX LIBPiARY L , .i “OX LIBRARY 

NAVAL POSTGRADUATE SCHOOL ' ' -TGRADIIATE SCHOOL 

MONTEREY, CA 93943-5101 



ABSTRACT 

A detailed analysis of the performance the Wiener optimal filter for estimating a 
signal in additive noise is carried out. A first order AR model is assumed for both the 
signal and noise. Both HR and FIR forms of the filter are considered and expressions are 
derived for the processing gain, mean-square error and signal distortion. These measures 
are plotted as a function of the model parameters. 

This analysis motivates a generalized form of the Wiener filter, which can improve 
the signal distortion. An analysis of this more general filter is then carried out. 

A practical noise removal algorithm based on short-time filtering using the gen- 
eralized filter is also described, and results of applying the algorithm to some typical 
underwater acoustic data are presented. 



V 



I 






I • 



« ^ • 









-m 




vi 







TABLE OF CONTENTS 



I. INTRODUCTION 1 

A. MOTIVATION FOR THE STUDY 1 

B. PROBLEM DESCRIPTION 2 

1. Problem Statement 2 

2 . Solution of The Wiener-Hopf Equations: 

The HR Filter 4 

3 . Solution of The Wiener-Hopf Equations: 

The FIR Filter 5 

C. ANALYSIS MODELS 7 

D. FILTERING AND MEASURES OF PERFORMANCE 9 

E. THESIS OUTLINE 11 

II. ANALYSIS OF THE HR WIENER FILTER 13 

A. THE HR WIENER FILTER 13 

B. PROCESSING GAIN FOR THE HR FILTER 20 

C. MEAN SQUARE ERROR FOR THE HR FILTER 24 

D. SIGNAL DISTORTION FOR THE HR FILTER 27 

E. SUMMARY 31 

III. ANALYSIS OF THE FIR WIENER FILTER 35 

A. THE FIRST ORDER FIR WIENER FILTER 35 

1. Processing Gain for First Order Filter 38 

2. Mean Square Error for First Order Filter 41 

3. Signal Distortion for First Order Filter 43 

B. PERFORMANCE MEASURES FOR HIGHER ORDER FIR WIENER 

FILTERS 45 

1. Processing Gain for Higher Order FIR Filter 46 

vii 



2. Mean Square Error for Higher Order FIR Filter 49 

3. Signal Distortion for Higher Order FIR Filter 51 

C. SUMMARY 52 

IV. THE EXTENDED OPTIMAL FILTER 57 

A. THE EXTENDED OPTIMAL FILTER PROBLEM 57 

B. PERFORMANCE OF THE GENERALIZED FILTER 59 

V. APPLYING EXTENDED OPTIMAL FILTERING TO UNDER- 
WATER SIGNALS 63 

A. INTRODUCTION 63 

B. RESULTS 65 

VI. CONCLUSIONS 73 

A. THESIS SUMMARY 73 

B. SUGGESTIONS FOR FUTURE RESEARCH 74 

LIST OF REFERENCES 75 

INITIAL DISTRIBUTION LIST 77 



viii 



LIST OF FIGURES 



1. General Linear Signal Estimation Problem 3 

2. Power Spectral Density Function for Real Exponential Correlation Func- 
tion (a > 0 and a < 0) 8 

3. General Linear Signal Estimation Problem 9 

4. Location of Pole of the HR Filter as a Function of Signal and Noise Cor- 
relation Coefficients (a and 7) for Input Signal-to-Noise Ratio of 0 dB. . . 17 

5. Location of Pole of the HR Filter as a Function of a for 7 = 0 and Different 

Values of the Input Signal-to-Noise Ratio 18 

6. Location of the Pole of the HR Filter as a Function of a for 7 = —0.5 and 

Different Values of the Input Signal-to-Noise Ratio 19 

7. Comparison between Theoretical and Experimental Values of Processing 
Gain for the HR Wiener Filter as a Function of a, for 7 = 0 (White Noise) 

and Input Signal-to-Noise Ratio 0 dB 22 

8. Comparison between Theoretical and Experimental Values of Processing 

Gain for the HR Wiener Filter as a Function of a, for 7 = —0.5 and Input 
Signal-to-Noise Ratio —10 dB 23 

9. Processing Gain for the HR Wiener Filter as a Function of a for 7 = 0 and 

Different Values of Input Signal-to-Noise Ratio 24 

10. Processing Gain for the HR Wiener Filter as a Function of a for 7 = —0.5 

and Different Values of Input Signal-to-Noise Ratio 25 

11. Processing Gain for the HR Wiener Filter as a Function of a for 7 = —.999 

and Different Values of Input Signal-to-Noise Ratio 26 

12. Comparison between Theoretical and Experimental Values of Mean Square 
Error for the HR Wiener Filter as a Function of a for 7 = 0 (White Noise) 

and Input Signal-to-Noise Ratio of 0 dB 27 



IX 



13. Mean Square Error for the HR Wiener Filter as a Function of a for 7 = 0 

(White Noise) and Different Values of Input Signal-to-Noise Ratio 28 

14. Mean Square Error for the HR Wiener Filter as a Function of a for 7 = 

—0.5 (Colored Noise) and Different Values of Input Signal-to-Noise Ratio. . 29 

15. Comparison between Theoretical and Experimental Values of Signal Dis- 

tortion for the HR Wiener Filter as a Function of a for 7 = 0 and Input 
Signal-tO-Noise Ratio 0 dB 30 

16. Signal Distortion for the HR Wiener Filter as a Function of a for 7 = 0 

(White Noise) and Different Values of Input Signal-to-Noise Ratio 31 

17. Signal Distortion for the HR Wiener Filter as a Function of a for 7 = —0.5 

(Colored Noise) and Different Values of Input Signal-to-Noise Ratio 32 

18. Location of the Zero of the First Order FIR as a Function of the Signal and 
Noise Correlation Coefficients (a and 7 ) for Input Signal-to-Noise Ratio of 

0 dB 37 

19. Location of the Zero of the First Order FIR as a Function of the Signal and 
Noise Correlation Coefficients (a and 7 ) for Input Signal-to-Noise Ratio of 

-10 dB 38 

20. Location of the Zero of the First Order FIR Filter as a Function of a for 

7 = —0.5, and for Different Values of the Input Signal-to-Noise Ratio. ... 39 

21 . Comparison between the Theoretical and Experimental Values of the Pro- 

cessing Gain for the First Order FIR Filter for 7 = 0 (White Noise) and 
Input Signal-to-Noise Ratio {pin) of 0 dB 40 

22 . Comparison between the Theoretical and Experimental Values of the Pro- 
cessing Gain for the First Order FIR Filter for 7 = 0.5 (Colored Noise) 

and Input Signal-to-Noise Ratio [pin) of —10 dB 41 

23. Processing Gain for the First Order FIR Filter as a Function of a, for 7 = 0 

(White Noise), and Different Input Signal-to-Noise Ratio 42 



X 



24. Comparison between the Theoretical and Experimental Values of the Mean- 

Square Error for the First Order FIR Filter for 7 = 0 (White Noise) and 
Input Signal-to-Noise Ratio of 0 dB 43 

25. Comparison between the Theoretical and Experimental Values of Mean- 

Square Error for the First Order FIR Filter as Function of a. for 7 = 0.5 
(Colored Noise) and Input Signal-to-Noise Ratio of — 10 dB 44 

26. Mean-Square Error for the First Order FIR Filter as a Function of a for 

7 = 0.5 (Colored Noise) for Different Values of Input Signal-to-Noise Ratio. 45 

27. Comparison between the Theoretical and Experimental Values of Signal 

Distortion for DiflPerent Values of a and for 7 = 0 (White Noise) at a Input 
Signal-to-Noise Ratio of 0 dB ' 46 

28. Signal Distortion for the First Order FIR Filter as a Function of a for 

7 = 0 and Different Values of the Input Signal-to-Noise Ratio 47 

29. Signal Distortion for the First Order FIR Filter as Function of o; for 7 = 0.5 

and Different Values of the Input Signal-to-Noise Ratio 48 

30. Processing Gain for the FIR Filter of Length P for 7 = 0 (White Noise) 

and Input Signal-to-Noise Ratio of 0 dB 50 

31. Mean-Square Error for the FIR Filter of Length P for 7 = 0 (White Noise) 

and Input Signal-to-Noise Ratio of 0 dB 51 

32. Signal Distortion for the FIR Filter of Length P for 7 = 0 (White Noise) 

and Input Signal-to-Noise Ratio of 0 dB 53 

33. Processing Gain for the HR Filter for 7 = 0 (White Noise) and Input 

Signal-to-Noise Ratio of 0 dB 54 

34. Normalized Mean-Square Error for the HR Filter for 7 = 0 (White Noise) 

and Input Signal-to-Noise Ratio of 0 dB 55 

35. Signal Distortion for the HR Filter for 7 = 0 (White Noise) and Input 

Signal-to-Noise Ratio of 0 dB 56 



XI 



36. Processing Gain for the Generalized Wiener Filter as Function of a and A 

for 7 = 0 (White Noise) and Input Signal-to-Noise Ratio of 0 dB 60 

37. Mean-Square Error for the Generalized Wiener Filter as Function of a and 

A for 7 = 0 (White Noise) and Input Signal-to-Noise Ratio of 0 dB .... 61 

38. Signal Distortion for the Generalized Wiener Filter as Function of a and 

A for 7 = 0 (White Noise) and Input Signal-to-Noise Ratio of 0 dB 62 

39. Prewhitening in Short-Time Extended Filtering Algorithm. 64 

40. Overlap Averaging Technique Used in Noise Removal 65 

41. Results of the Application of the Extended Optimal Filtering Technique 

to a Synthetically Generated Short Pulse Signal with Added Low Power 
White Noise, for Minimum Residual Noise (A = 1). (a) Original Clean 
Data, (b) Original Data plus White Noise, (c) Processed Data 68 

42. Results of the Application of the Extended Optimal Filtering Technique 

to a Synthetically Generated Short Pulse Signal with Added Low Power 
White Noise, for Residual Noise Power Equivalent to 30% of the Input 
Noise Power, (a) Original Clean Data, (b) Original Data plus White 
Noise, (c) Processed Data (d) Values of A for Each Segment 69 

43. Results of the Application of the Extended Optimal Filtering Technique 

to a Synthetically Generated Short Pulse Signal with Added Low Power 
White Noise, for Residual Noise Power Equivalent to 70% of the Input 
Noise Power, (a) Original Clean Data, (b) Original Data plus White 
Noise, (c) Processed Data (d) Values of A for Each Segment 70 

44. Results of the Application of the Extended Optimal Filtering Technique 

to a Killer Whale Song for Minimum Residual Noise (A = 1). (a) Original 
Noisy Data, (b) Processed Data 71 

45. Results of the Application of the Extended Optimal Filtering Technique 

to a Killer Whale Song for Residual Noise Power Equivalent to 20% of the 
Input Noise Power, (a) Original Noisy Data, (b) Processed Data 72 

xii 



ACKNOWLEDGMENTS 



The author thanks Father in Heaven for all the bestowed blessings that made this 
work possible. 

Thanks to my wife, Cristiane, and my daughters, Beatriz and Fernanda, for the 
love, patience and support. Without them, there would not be much motivation to 
accomplish such challenging task. 

Also, thanks to Professor Charles W. Therrien for his support and encouragement 
during all the research process. 



xiii 










XIV 



I. 



INTRODUCTION 



A. MOTIVATION FOR THE STUDY 

Data from passive sonar is generally accompanied by ambient noise arising from 
shipping traffic, marine life, wave motion, moving or cracking ice (in the Arctic and 
Antarctic), and numerous other sources. The statistical properties of the noise are vari- 
able, even direction-dependent, and have been the source of many studies and analyses 
[Refs. 1, 2, and 3]. Noise degrades sonar data collection and related processing of the 
data to extract information. 

Since ambient noise cannot be completely avoided when collecting real underwa- 
ter acoustic data, it is desired, in many situations, to remove the noise before further 
processing. In general, the signals of interest and the ambient noise are non-stationary 
and their statistics are not known a priori. Here we approach the problem of removing 
additive noise from a given signal by using a short-time optimal filtering technique. In 
this approach we assume that the noise is stationary for the duration of the signal, and 
that the signal can be assumed stationary over very short time intervals. We exploit this 
feature and develop improved algorithms for removing the noise. 

The work in this thesis basically consists of two parts. In the first part (Chapters 
II and III), we perform a detailed analysis of the Wiener filter and evaluate its perfor- 
mance using three different criteria. For this work we use a simple signal model which 
nevertheless provides insight into results for more general types of signals. Both the HR 
and FIR forms of the filter are considered. In the second part (Chapters IV and V), we 
propose a new algorithm based on a generalization of the Wiener filter and motivated by 
our analysis, that can be applied to real data. The implementation of the algorithm on a 
short-time basis is similar to that developed by Frack [Ref. 4] and we use the same basic 
structure of the programs. Results of the application of this algorithm to underwater 
acoustic data (biologic data) are presented. 
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B. PROBLEM DESCRIPTION 

In this section we describe the problem of noise removal to be addressed and 
introduce the Infinite Impulse Response (HR) and the Finite Impulse Response (FIR) 
forms of the Wiener filter used to estimate the signal in noise. 

1. Problem Statement 

We consider here the problem of estimating a signal in additive noise. The ob- 
served discrete observation sequence is given by 

x{n) = s(n) -1- T]{n) (I.l) 

where s(n) is the signal and r}{n) is the noise. The signal duration may be anywhere 
from a few miliseconds to a few seconds and is generally a nonstationary random process. 
The noise is assumed to be stationary over the entire observation interval and may be 
observed without the signal during the early part of the observation interval. That is, 
the signal becomes non-zero some time after the beginning of the observation interval, 
and the precise time at which it becomes non-zero is not known. Statistics for the signal 
and noise, such as mean or correlation function, are not known a priori and must be 
estimated from the data. 

Let us assume that the signal is to be estimated by applying the observation 
sequence to a linear filter as shown in Figure 1, and the filter is designed to minimize the 
mean-square error 

where s(n) = y(n) is the filter output. Then the solution for this problem is obtained by 
solving the Wiener-Hopf equation [Ref. 5], which for the case of a stationary signal has 
the form 

OO 

^ — i)h[l] = Rsi(i); 0 < i < oo (1.2) 

1=0 

Here Rx{l) is the correlation function of the observations, h{l) is the impulse response 
of the filter, and Rsx{l) is the cross-correlation function between the signal and the 
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x{n) = s(n) + Tj{n) 



H(z) 



y[n] = s(n) 



Figure 1. General Linear Signal Estimation Problem. 



observation sequence. Since the noise is assumed independent of the signal, it is easy to 
show that 

i?x(0= ^.(0 + ^(0 (1-3) 

and 

Rs.{l) = Rs{l) (1.4) 

where Rs{l) and Rr]{l) are the signal and noise correlation functions, respectively. The 



estimate for the signal is then given by 

n 

${n) = y{n) = ^ h{n — k)x{k) (1.5) 

k=~~oo 

and the mean-square error can be shown to be 

OO 

al = Rs{0)-Y.h{l)RsAl) ( 1 - 6 ) 

1=0 

where again Rsx{l) = Rs{i) for this problem. 

When the signal s(n) is nonstationary, as is generally the case in our application, 
the optimal filter is time-varying and equations (1.2) through (1.6) take on a slightly 
different form [Ref. 5] involving the time varying correlation functions Rs{ni,Ti 2 ) and 
Rx{ni,n 2 ). Although these correlation functions could be estimated from the data and 
the corresponding time- varying filter could be computed, it is generally more efficient to 
take a block-oriented approach where the data is assumed stationary over short segments 
and the equations for a time-invariant filter are used to find a filter that applies only to 
the data segment. Although we do not do so in this work, the data segments could be 
overlapped by some amount, and indeed when the overlap consists of all but the a single 
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point, the result is equivalent to using the time-varying filter. Thus for purposes of our 
work and analysis here, (1.2) through (1.6) are the relevant equations. 

The Wiener-Hopf equation (1.5) can be solved for two types of filters, the Infinite 
Impulse Response (HR) filter, which is applied recursively to the data, and the Finite 
Impulse Response (FIR) filter, where the filtering process involves a simple convolution. 
Both of these filters are known as Wiener optimal filters and are described in following 
sections. Later, we investigate the performance of these two filters in the case when both 
signal and noise are represented by a first order autoregressive (AR) model. This inves- 
tigation using a simple model provides insight into the general behavior of the Wiener 
Filter, and so, helps to predict its behavior for more involved cases. 



2. Solution of The Wiener-Hopf Equations: 
The HR Filter 



The HR Wiener filter is recursive in form and can have the advantage of requiring 
fewer parameters than a comparable FIR form of the filter. In many cases the IIR filter 
is the only true optimal solution to the problem and FIR filters are in fact suboptimal 
since they do not achieve the absolutely lowest mean-square error. Since the filter is 
required to be causal, the problem is not straightforward; it was solved (originally in the 
continuous case) by Wiener using spectral factorization methods [Ref. 6]. Because the 
solution procedure will be referred to later in this work, we summarize it in this section. 

The solution for the IIR filter derives from observing that if the observation se- 
quence is a white noise process Rx{l) = <JqS{1) then (1.2) becomes 

OO 

X3<t^5(Z - i)h{l) = Rsx(i); 0 < f < OO (1.7) 

1=0 

leading to the simple solution 



The procedure then is to first whiten the observed random process x and then apply the 
solution above. Thus we conceptually represent the optimal filter /i as a cascade of two 



■^Rsx{l) I > 0 



I < 0 



( 1 . 8 ) 
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filters, the whitening filter g, and the optimal filter K for the whitened process. Both g 
and Ji are required to be causal. 

To find the whitening process we observe [Ref. 5] that the complex spectral density 
function of the input process x can be factored as 



S^{z) = ]CoHca{z)Hca{z-'^) 



(1.9) 



Therefore, if we choose G{z) = 1/Hca{z), we see that the filter g will have the necessary 
whitening properties and the whitened input process will have variance /Cq. 

The cross-correlation between the whitened input process and the signal has a 
complex cross-spectral density function given by Ssx(z)/Hca{z~^); therefore, the filter 
corresponding to (1.8) for the whitened process is written as 



H'(^) = I- 



Ssx{z) 



JCo [Hcaiz-'^) 



(I.IO) 



where the notation [ ]+ means that the resulting function corresponds only to the causal 
part of the quantity within brackets. Finally, by cascading G(z) and H'{z) we obtain the 
expression for the optimal HR filter 



JJ(~\ — ^ 


Ssx(z) 




lCoHca(z) 


[Hcaiz-^)\ 


+ 



(I.ll) 



where Hca(z) and ICq are derived from the spectral factorization (1.9), and Ssx{z) is the 
cross spectral density function for s(n) and x{n). 



3. Solution of The Wiener-Hopf Equations: 

The FIR Filter 

The FIR filter is simpler to derive. Since the filter has finite length P, the filtering 
equation (1.5) can be written as 

y{'^) = Y. - 0 ( 1 - 12 ) 

1=0 

We can also directly apply (1.2) noting that the upper and lower limits need to reflect 
the causality and finite length of the filter. The Wiener-Hopf equation for real process 
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then becomes 



p-\ 

Y, Rx(l - i)h(l) = Rsx{i)\ z = 0, 1, . . . , P - 1 (1.13) 

z=o 

and the mean-square error (1.6) is given by 

a? = RM - h{l)R,M (1.14) 

/=0 

For the FIR filter it is most convenient to write the equations using vector nota- 
tion. The filter output is written as 



y[n] = h^x 



(1.15) 



where h and x are the vectors of filter coefficients and observations respectively 



ft(0) 




a:(n) 


Ml) 




x{n — 1) 


A(P-l) 


X = 


x(n — P + 1) 



Equations (1.13) and (1.14) can then be written as 



Ra;h = Tsx (1.16) 

and 

al = Rs{0) - (1.17) 

where Rj; = E |x(n)x^(n)| and Tgx = E {s(n)x(n)}. Since the signal and noise are 
uncorrelated, we have 

Re = R« + R77 ( 1-16) 

where Rj and R,, are the signal and noise correlation matrices, and r^x is the first column 
of R5. This form of the equations is used in the analysis that is presented later in this 
thesis. 
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C. ANALYSIS MODELS 



The detailed analyses presented in this thesis are based on a first order autore- 
gressive (AR) model for the signal and the noise. The biological signals and the ocean 
noise tend to have a Gaussian distribution. The AR model is a linear model, so when 
modeling these signals and noise it preserves their Gaussian nature. Also, the solution 
of AR model depends only on a system of linear equations, allowing a minimum phase 
solution, i.e., providing a causal stable model with a causal stable inverse [Ref. 5]. The 
signal s{n) is assumed to be generated according to the difference equation 



s{n) = as{n — 1) + Ws{n) 



(1.19) 



where Ws{n) is a (zero mean) white noise process with variance <7^^ and o; is a real valued 
parameter with |q:| < 1. The correlation function for the signal is then found to be 

R,(l) = cr|al" (1.20) 



where the signal power <7^ is given by 



2 _ 



o. = 



1 — Q;^ 



( 1 . 21 ) 



Thus a is seen to represent the correlation coefficient for the process {a = i?(l)/i?(0)) 
with higher values of a representing increased correlation. The complex spectral density 
function for the signal is the z-transform of the correlation function Rs{l) and has the 
form 



Ss{z) = 



(1 - « 2 ) 



(1.22) 



(1 — az){l — az~^) 

If this is evaluated on the unit circle {z = we have the power spectral density function 



s (e^n = 

’ l+a^-2c,cos{u>) 



(1.23) 



For q ; > 0 this spectrum has a lowpass character while for a < 0 the spectrum has a 
highpass character (see Figure 2). For a = 0 the signal is a white noise sequence and the 
spectrum is flat. 
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Figure 2 . Power Spectral Density Function for Real Exponential Correlation Function 
(a > 0 and a < 0 ). 



The noise 7 /(n), when it is not white, is represented by a similar model. The noise 
satisfies the difference equation 

r)(n) = 7 ? 7 (n - 1) + Wr,{n) (1-24) 



where Wjj{n) is a white noise process independent of Wj(n), and 7 is real with magnitude 
less than 1 . The process has a correlation function of the form 





(1.25) 


where the noise power cr^ is given by 




< 

^ 1-72 


(1.26) 


and a complex spectral density function 




(1 - 7^) 

(1 - 7^)(1 - 7-^-^) 


(1.27) 
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x{n) = s(n) + T](n) 



H{z) 



yip) = y,(n) + y,(n) 



Figure 3. General Linear Signal Estimation Problem. 



The power spectral density function is therefore given by 

s (ei") = - t') 

1 + 7 ^ — 2'ycos(u) 



(1.28) 



and has a lowpass or highpass character according to the sign of 7 . As described earlier, 
the noise is assumed to be added to the signal, and the main problem addressed in this 
thesis is the removal of the noise or estimation of the signal. 



D. FILTERING AND MEASURES OF PERFORMANCE 

We have considered the problem where we observe the sequence x(n) given in (I.l) 
where s(n) is the desired signal and T)(n) is additive noise uncorrelated with the signal. It 
is desired to remove as much of the noise as possible by linear filtering. Although we have 
already discussed the optimal solution in terms of mean-square error, let us temporarily 
put that solution aside and formulate the problem more generally. We assume that the 
observed sequence x(n) is applied to a filter and the output of the filter y(n) is an estimate 
of the signal s(n). Since the filter is linear, we can define components of the output as 
shown in Figure 3. In particular ys(n) is the response of the filter to the signal alone and 
yT)(n) is the response to the noise alone. These components permit defining the following 
measures of performance: 

Processing Gain: The processing gain (PG) for the filter is defined as 



PG = 



pout 

pin 



where pin and Pout ^tre the input and output signal-to-noise ratios defined as 

R,(0) E{s2(t>)} 

H,(0) E{r,=(n)} 



(1.29) 



(1.30) 
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(1.31) 



R,.{0) E {</?(»»)} 
E{!/J(n)} 



The processing gain is usually measured in dB, i.e., 



PG(dB) = 10 log , 0 



pout 

pin 



(1.32) 



Processing gain relates to the relative power in the signal and the noise but does not 
address accuracy of the signal estimate. 

Mean Square Error: The mean-square error is the estimated value of the 
squared difference between the signal and the filtered output. Here we use a normal- 
ized measure of the mean-square error defined by 

E{(s(n)-s,(n))n R,{0) 



MSE = 



(1.33) 



E{s2(„)} R,{0) 

where Re{l) is the correlation function for the “error” defined as e(n) = s(n) — s(n) = 
s(n) - y{n). 

When the output of the filter is equal to the signal itself (except possibly at a 
countable set of points) then the mean-square error is equal to zero. The maximum 
occurs when y(n) is zero, where this normalized mean-square error has a value of 1. 
Mean-square error is the quantity that is optimized in the traditional Wiener optimal 
filtering problem discussed above. 

Signal Distortion: Signal distortion (SD) addresses what the filter does to the 
signal itself irrespective of the noise, and is defined as 

E {s{n)ys{n)}^ (0) 



SD = 1 - 



= 1 - 



(1.34) 



E{s2(n)}E{y2(n)} E,(0) E,,(0) 

where Rsys(l) is the cross-correlation function between s(n) and ys(n). This quantity 
takes on values in the interval [0, 1] and is minimized when the signal is unchanged by 
the filter (ys(n) = s(n)), i.e., there is no distortion introduced. 

An ideal filter would form a perfect estimate for the signal and completely elimi- 
nate the noise. Such a filter would have infinite processing gain, zero mean-square error. 
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and zero signal distortion. Since no linear filter can achieve such ideal performance, 
we can balance these measures, depending on the application, emphasizing one measure 
compared to the others. Thus all of the measures defined above will have use in the 
sequel. 

E. THESIS OUTLINE 

The remainder of this thesis is organized as follows. Chapter II introduces, anal- 
yses and discusses the performance of the HR Wiener filter described above. Chapter 
III provides a similar analysis and discussion of the performance for the FIR form of 
the Wiener filter. Chapter IV presents a filter based on a new criterion that serves as a 
generalization of the FIR Wiener filter. This filter, which we call the extended optimal 
filter, provides a way to improve the signal distortion with only slight degradation in the 
other performance measures. Chapter V incorporates this technique in a noise removal 
algorithm based on short term filtering of the observation sequence and demonstrates 
its performance on sonar data. Chapter VI concludes the thesis with a summary and 
suggestions for further investigation. 
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II. 



ANALYSIS OF THE HR WIENER FILTER 



In this chapter we derive analytical expressions for the Wiener HR optimal filter 
for a signal in colored noise. The signal and the noise are each represented by a first 
order AR model (equations (1.19) and (1.24)). The expressions for the filter generalize 
the results obtained in [Ref. 5] which treats the white noise case only. 

After deriving an expression for the optimal filter we derive the corresponding 
expressions for the measures of performance outlined in Chapter I and examine these 
expressions as a function of some of the parameters describing the model. While the 
model expressed by (1.19) and (1.24) is only of first order, it exhibits many of the gen- 
eral characteristics of a more complicated model and allows us to draw several general 
conclusions. 



A. THE HR WIENER FILTER 

The general expression for the optimal HR (Wiener) filter for estimating a real- 
valued signal s(n) in additive noise r][n] (see (I.l)) is given by equation (1. 11). For the 
case of a first order AR model, the signal and noise complex spectral density functions 
are given by (1.22) and (1.27) respectively. Since the signal and noise are independent 
with zero mean, it follows from (1.4) that the complex spectral density function between 
the signal s(n) and the observation x{n) is given by 



Ssx{z) = Ss(z) = 



(II.l) 



(1 — oiz){l — az~^) 

Similarly, it follows from (1.3) that the complex spectral density function for the obser- 
vations is 



Sx{z) = Ss{z)+Sr,{z) 

(1 — 0!z)(l — az~^) (1 — 7z)(l — 'jz~^) 

_ (7s(l — |o!p)(l -I- 7^ - 7z — 7z“^) - 1 - cr^(l - l7p)(l + — az — 0 'Z~^) 

(1 — az~^){l — az){\ — 72:“^)(1 — 72) 
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By collecting terms, this last equation can be written as 



where 



Sx{z) 



—Az + B — Az~^ 

W) 



A = - a^) +Q!(T^(l-j^) 



and 

D{z) = (1 — Q! 2 r“^)(l — az){l — 7 z”^)(l — 72 ) 



After factoring the numerator, we can write (II.2) as 



Sx{z) = 






(z - Zij(z - Zii.) 



where 



Zla = \ (C-VC^) 

zn = \ (C + VC^) 



and 

B ^ <jj{\ - Q^)(l + 7^) + g^(l - 7^)(1 + a^) 
A <7|(1 - a2)7 + cr2(l - 7 ^) 0 ; 

This last constant can be expressed as a function of <x, 7 , and pi„, as 



^ [pin(l - Q^)(l + 7^)/(l - 7^)] + (1 + a^) 

\pinl{l - a^)/(l - 7^)] + a 



where pin is the input signal-to-noise ratio 




(11.2) 

(11.3) 

(11.4) 

(11.5) 

(11.6) 

(11.7) 

(11.8) 

(11.9) 

( 11 . 10 ) 

( 11 . 11 ) 
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Zi = 



From the form of the polynomial in (II. 2) and (II. 6), it can be seen that ziaZ\h = 1 , or 
that Zi6 = l/^ia. Let us denote the root that is inside the unit circle in the complex 
plane as z\, i.e., 

ifkuKi 

zib = l/z\a Otherwise 
Then from (II. 10 ), we see that zi depends only on a, 7, and pin- 

We can now factor the complex spectral density function S'* (2:) by writing it in 
the form 



S,{z) = - 



(1 - Ziz) 



zi {1 — Qz~^){l — 'yz~^) {l — az){l—^z) 
In comparing this with ( 1 . 9 ) we can identify 

/Co = Af z\ 

(1 - 



(11.14) 



and 



Hca{z) = 



(1 — az“^)(l — 7z~^) 
Ssx{z) 



[Hca{z-^)\ 



Let us now proceed to compute the term 
equation for the optimal filter. From (II. 1) and ( 11 . 15 ), we have 

Ssxjz) ^ a,^(l-|gp) (l-az)(l-7;^) 

Hca{z~'^) (1 - q;z)( 1 - Q!Z“1) (1-Ziz) 

= „|(1 _ I ^ P ) 21 (1 - ( 1 / 7 )^-') 



(11.15) 

that appears in (1. 11), the 



ziz (1 — az~^)(l — (l/zi)z“^) 
This can be written in a partial fraction expansion as 

Ssxiz) _ Cl 



where 



and 



Cl = (7^(1 - |a|2) 

C2 = ^2(1 - lap) 



.+ 

(1 - (iM)z-i) 


(11.16) 


7 1 - (l/a7) 
zi (1 - l/zicx) 


(11.17) 


7 1 - (zi/7) 
zi (1 - zia) 


(11.18) 
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The causal part of the expression (11.16) can now be written as 



H\z) 



ICo 



Ssx{z) 

Hca{z-^) 



Cl 

1 — az~^ 



(IU9) 



^ 1 - 1/aj (j^(l - lap) 7/zi 

A 1 — 1/zia 1 — az~^ 



£i_ a - 1 /j cr^(l - lap) j/zi 
A a — l/z\ 1 — az~^ 



Finally, combining (11.15) and (11.20) we obtain 



ff(z) 



1 1 
ffca(z) JCo 



Ssxjz) 

Hcaiz-^) 



+ 



( 11 . 20 ) 



{ 1 -az ^)(l-7z ^) £i a - I/7 g^(l - [ap) j/zi 
{1 — ziz~^) A a — lfzi (1 — Q!Z“^) 



/ zi azi - zi/7 2 

y A azi — 1 



(1 - lap) 7/zi 



-1 



1 — 7Z 
l-ZiZ-\ 



( 11 . 21 ) 



The final expression for the Wiener optimal HR filter can thus be written as 



ff(z) = G 



-1 



1 — jz 



( 11 . 22 ) 



where the gain G is given by 



A azi — 1 



(11.23) 



It is interesting to note that the filter places a zero at 2 = 7 in an attempt to cancel the 
additive noise. Also, it is important to note that Zi is just a function of the constant C, 
and, therefore, through (II. 10 ), it is just a function of the input signal-to-noise ratio Pin, 
the signal parameter a, and the noise parameter 7 . 
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The following figures show the behavior of the pole Zi for different situations. In 
Figure 4, the input signal-to-noise ratio (pin) is fixed, but the signal parameter a varies 
from —1 to +1. We have plotted the location of the pole zi of the optimal filter for 
different values of 7 (7 = 0 represents the white noise case). Notice that when a = 7, the 
filter places the pole at the same location (zi = 7), since it cannot distinguish the signal 
form the noise. Also observe that when a = ±1, the pole is placed at z = ±1, regardless 
of the value of 7. In Figure 5, the noise parameter 7 has a fixed value of 0 corresponding 




Figure 4. Location of Pole of the IIR Filter as a Function of Signal and Noise Correlation 
Coefficients (a and 7) for Input Signal-to-Noise Ratio of 0 dB. 

to white noise, and the pole Zi is plotted for different values of the input signal-to-noise 
ratio. Notice in Figure 5 that when the input SNR is small, the pole of the IIR filter 
approaches the signal parameter a. This can be shown analytically as follows. 
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signal correlation coefficient (alpha) 



Figure 5. Location of Pole of the HR Filter as a Function of a for 7 = 0 and Different 
Values of the Input Signal-to-Noise Ratio. 



First, from (II. 10) we see that 



/ I 

lim (C) = 

Pin-^O OC 



Thus, 



lim (zi ) = 



1 / (1 + a^) /(I + “ 40:^ 



a 



Q,Z 



— —{2a^) = a 
2a^ ’ 



(11.24) 



In a similar manner, we can show that for a large input SNR, the pole approaches the 
noise parameter 7 . We have 

1 + 

lim (C) = 

Pin->00' ^ 
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signal correlation coefficient (alpha) 

Figure 6 . Location of the Pole of the HR Filter as a Function of a for 7 = —0.5 and 
Different Values of the Input Signal-to-Noise Ratio. 

Therefore 



lim zl 

Pin-^00 



1 /(l+T^) , /(I + 7^)2 - 47^ 

2 \ 7 V 7 ’ 

= T 



(11.25) 



This characteristic can also be observed in Figure 6 , where we have plotted the location 
of the HR filter pole as a function of a, for different values of the input signal-to-noise 
ratio, and for 7 = —0.5. As we will see in a later section, this behavior leads to a better 
performance, in terms of the processing gain, for smaller values of the input signal-to- 
noise ratio. 
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B. PROCESSING GAIN FOR THE HR FILTER 



Using the first order signal and noise model, we can derive an analytical expression 
for the processing gain (1.29) for the HR Wiener Filter. For this we first need the input 
and output signal-to-noise ratios. The input signal-to-noise ratio is given by (1.30) or 
(II.ll), while the output signal-to-noise ratio is given by (1.31). Hence, the problem is 
to find expressions for i?j,^( 0 ) and Rj,„( 0 ) in terms of a, 7 , cr^, and To do this, first 
observe that the complex spectral density of the output of a linear system is given by 
[Ref. 5] 

5y(^) = H{z)H{z-^)S^[z) (11.26) 

where H{z) is the filter transfer function and Sx is the complex spectral density function 
of the input. Since the correlation function is related to the complex spectral density 
function by the inversion formula 

RyO) = 

= ^ Residues [5y ( 2 ) 2 *“^] 

where the residues correspond only to the poles within the unit circle, we have 



Ry(0) — Residues[ 5 y( 2)2 

= y)Residues[H(2)H(2“^)5a;(2)2“^] 



(11.27) 



To evaluate (0) we use this formula assuming that the input complex spectral density 
function Sx(z) is that of the signal. Thus from (11.26), (1.22), and (11.22) we obtain 

72 



„ . , , ^9 1 1 — 72 ^ 1 

Sy,W 2 -‘=G^- — 



2i2“^ ' 1 — 2i2 (1 — Q!2“^)(l — az) 



(11.28) 



and, according to (11.27) we have 



Ryj (0) — RESz=2j -I- RESz=q -I- RES2=o 



(11.29) 



where the three residues are given by 

G^(l - 7Z-')(1 



RESz=zi ~ 



(2 — 2i)(l — 2i2)(l — Qf2)(l — az~^) 






z=zi 
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GV^(1 - a^)(l - 7 Ai)(1 - 7 ^ 1 ) 

(1 - zl){l - a/zi)(l - azi) 



RES^^ck 



(?{l-iz ^)(l- 7 z)a^(l-g^) _ 

(1 — ziz~^){l — z\z){l — az){z — a) 
GV,^(1 -7/g)(l-7Q:) 

(1 - zi/a){l - zia) 



RFS = G"(i - 7 ^ ^)(i - (1 - . X 

z(l — ~ — az){l — az~'^) ^ 



= 0 



2=0 



Thus combining these results, we have 

RyX ^) = < jlG ^ 



(1 - 7/^1) (1 - 72i)(l - (1 - 7 /a)(l - 7a) 



(11.30) 



(1 - 2^)(1 - a/zi){\ - azi) (1 - ^i/g)(l - zia)_ 

In a similar manner we see that the output complex spectral density due to the noise 
alone is 

,-i 



c (^\ - ' 1-7^ ^^(1-7^) 



= GV 



2^2 



(1 - 7 ^) 



(11.31) 



’'(1 -ZlZ-l)(l - Ziz) 

Applying (11.27) and evaluating the expression at the single pole z = zi then leads to 

1 






(11.32) 



By taking the ratio of (11.30) and (11.32) we compute the output signal-to-noise 



ratio 



(1 -7/zi)(l -7Z1) (l-7/g)(l-7a)(l-Zi2) 

Pout = r^rr: r + 



(l-a/zi)(l-o,^.) ' (1 - 2,/a)(l - zic)(l - a2) 

Finally, by substituting (II. 11 ) and (11.33) into the definition (1.29) we obtain the final 
expression for the processing gain as 



PG = 


■(1 -7/zi)(l -7Z1) (1 -7/g)(l-7a)(l -zf) ■ 


1 — 




(1 - g/zi)(l - azi) (1 - 2i/g)(l - zia){l - a^) _ 


1 — 72 





(11.34) 



Figure 7 shows the theoretical and experimental values of the processing gain for 
different values of a, and for 7 = 0 , representing additive white noise, at a signal-to-noise 
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ratio of 0 dB. Figure 8 shows the same curves, but for 7 = —0.5 (colored noise) at an 
input signal-to-noise ratio of —10 dB. The experimental curve was obtained by averaging 
measured results for 10 different trials for each value of a, in order to obtain a smaller 
curve variance. Notice that there is very good agreement between the theoretical and 
experimental values. 




Figure 7. Comparison between Theoretical and Experimental Values of Processing Gain 
for the HR Wiener Filter as a Function of a, for 7 = 0 (White Noise) and Input Signal- 
to-Noise Ratio 0 dB. 

Observe from Figures 7 and 8 that the processing gain for the first order IIR filter 
has its maximum values at a = ± 1 . We can examine the behavior explicitly at these 
limits as follows. For a — ±1 the constant C in (II. 10) approaches ± 2 . It follows then 
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Figure 8. Comparison between Theoretical and Experimental Values of Processing Gain 
for the HR Wiener Filter as a Function of a, for 7 = —0.5 and Input Signal- to-Noise 
Ratio —10 dB. 



from (II.7) and (II.8) that the root Z\ approaches ±1. 



lim. 






PG = 



(1 - zi)(l - zi) 1 - 72 



Using this result in (11.34) we find 

(l-7 )(l- 7 )(l-zi)(l+ 2:i) 

(1 - zi)(l - zi)(l - 7)(1+ 7) 



(l-7)(l-hzi) 

(1 -zi)(l + 7) 



(11.35) 

Figures 9 through 11, show the behavior of the processing gain for different values 
of the input SNR {pin), and for 7 = 0, 7 = —.5, and 7 = —.999. Notice that in all 
of these cases the processing gain approaches infinity as o; ^ ±1. Intuitively, we can 
see from (11.22) that, for a completely correlated signal {a = ±1), the optimal HR filter 
attempts to null the noise (by placing a zero at 7) while completely let the signal pass 
(pole at zi = a = ±1). Thus, while the input signal-to-noise ratio remains finite, the 
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output signal-to-noise ratio and thus the processing gain becomes infinite. 




Figure 9. Processing Gain for the IIR Wiener Filter as a Function of a for 7 = 0 and 
Different Values of Input Signal-to-Noise Ratio. 



C. MEAN SQUARE ERROR FOR THE IIR FILTER * 

An analytical expression for the mean-square error of the Wiener filter for a first 
order AR signal in white noise is given in [Ref. 5]. Here we extend that result to the 
general colored noise ca.se and derive an analytical expression for the MSE given by 
equation (1.33). 

If the error is defined as t{n) = s{n) — s{n), then (1.33) can be written as 

MSE = (11.36) 

where is the unnormalized mean-square error or error variance. It is shown in [Ref. 
5] that the complex cross spectral density function between the signal and the error is 
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Figure 10. Processing Gain for the HR Wiener Filter as a Function of o; for 7 = —0.5 
and Different Values of Input Signal-to-Noise Ratio. 



given by 

Sse{z) = 5 s ( 2:)(1 - H{z~^)) 
Substituting (1.22) and (11.22) then yields 



Sst{z) = 



- « 2 ) 



1 -G 



1 — 72: 
1 — Z]_z 



(11.37) 



(11.38) 



(1 — az){\ — az~'^) 

By the orthogonality theorem of linear mean-square estimation, the error variance is 
equal to the cross-correlation function i?se(0 evaluated at lag Z = 0. Therefore from the 
inversion formula for the complex cross spectral density we have 



gI = Rse(O) = ^ / Sst{z)z ^dz 

This integral, involving (11.38) can be evaluated by residues, giving 
^ Residues [5se(z)z''^] 



(11.39) 
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Figure 11 . Processing Gain for the IIR Wiener Filter as a Function of a for 7 = —.999 
and Different Values of Input Signal-to-Noise Ratio. 



= a 



(1 — Q'2)(l — az"*) V 1 — ZizJ 
V 1 — 21 a/ 



Finally, using this result in (11.36) we obtain 



MSB = 1 - G 



(1 - 7Q) 

(1 - zia) 



(11.40) 



Figure 12 compares the theoretical mean-square error for 7 = 0 and = 0 dB 
with the averaged measured squared errors for the same case. Again, there is a good 
agreement between the theoretical and the experimental results. Figure 13 shows the 
behavior of the mean-square error for input signal-to-noise ratio of —10 dB, 0 dB, and 
10 dB. Notice that, when a = 7 = 0 , MSB = + cr^)- For example, when pin = 0 
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Figure 12. Comparison between Theoretical and Experimental Values of Mean Square 
Error for the IIR Wiener Filter as a Function of a for 7 = 0 (White Noise) and Input 
Signal-to-Noise Ratio of 0 dB. 

dB, the power of the signal equals the power of the noise, and MSE = 0.5, as expected. 
The same analysis for = 10 dB results in MSE = 0. 1/1.1 « 0.1. 

Figure 14 shows a case when the noise is not white (7 = —0.5). The results are 
qualitatively similar to those for the white noise case. However the curves are skewed, 
indicating that the worse performance in terms of mean-square error occurs when the 
signal has a correlation coefficient equal to that of the noise. 

D. SIGNAL DISTORTION FOR THE IIR FILTER 

In this section we derive an explicit expression for the signal distortion (1.34) 
produced by the Wiener filter for the problem of a signal in additive noise. As in the 
previous sections, the signal and the noise are each represented by a first order AR model. 
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Figure 13. Mean Square Error for the HR Wiener Filter as a Function of o: for 7 = 0 
(White Noise) and Different Values of Input Signal-to-Noise Ratio. 



To evaluate the term in (1.34), we begin with the cross complex spectral 

density function between the input and output of the filter is given [Ref. 5] by 

Ssysiz) = H{z~'^)Ss{z) 

Therefore, as before, we can evaluate Rsy^ (0) from 
Rsysi^) = X^Residues[5j,,(2)z"^’ 

zizJ (1 



= Residues G ( ) y- y- 

^ \\ — Z\zJ (1 — az)(l — az ^ 



) 



The only residue is given by 



Residue = G t- — 



(1 - a 2 ) 



= 



(1 — z\z) ^ (1 — az){l — az~^)z~^(z — cc) 

(1 - 7Q) 

(1 - zya) 



(11.41) 



(11.42) 
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Figure 14. Mean Square Error for the IIR Wiener Filter as a Function of a for 7 = —0.5 
(Colored Noise) and Different Values of Input Signal-to-Noise Ratio. 



resulting in 






(11.43) 



Then, by substituting (11.43), (11.30), and i?s(0) = into (1.34) we can obtain an 
expression for signal distortion. 



SD = l- 



E 



where 



E = 



(1 - 7Q) 

(1 - zia) 



t2 



(11.44) 



(11.45) 



r, ^ (l-7/zi)(l -7Zi)(l -g^) (1 -7/g)(l -7a) 

[{l-zl){\-azi)[\-a/zi) Z]_/a){l- zxa)\ 

Figure 15 shows the theoretical expression (11.44) against the measured results as 

a function of the correlation factor g, for white noise, with an input signal-to-noise ratio 
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of 0 dB. Notice that for values of a close to ±1, i.e., for highly correlated signals, the 
distortion is very low. This is also apparent from Figure 16 when the signal distortion 




Figure 15. Comparison between Theoretical and Experimental Values of Signal Distortion 
for the IIR Wiener Filter as a Function of a for 7 = 0 and Input Signal-to-Noise Ratio 
OdB. 

is plotted for several different values of the input signal-to-noise ratio. Figure 16 also 
shows that the signal distortion is lower for higher values of the input signal-to-noise 
ratio, because the filter is able to pass more of the signal. Since we already know that 
lima_>.i zi = 1, we can evaluate the lima-).i for (11.46) and verify that lima-»i SD = 0. 
Also, for O' = 7 (which implies that H{z) is an all-pass filter) the distortion is zero, as 
expected. However, there is a value for a, which we shall call the critical value oic, where 
the distortion reaches its maximum value. This value depends on pin- 
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Figure 16. Signal Distortion for the HR Wiener Filter as a Function of o; for 7 = 0 
(White Noise) and Different Values of Input Signal-to-Noise Ratio. 

Figure 17 shows the result for 7 = —0.5. Again the distortion peaks at specific 
critical values of the correlation coefficient ac- However, in this case the two values do 
not occur at locations symmetrically distributed about the origin. 

E. SUMMARY 

The analysis of this chapter based on the first order AR signal and noise model 
shows that the optimal filter for estimating the signal is also of first order. The filter has 
a zero at the location 7 corresponding to the noise correlation coefficient, and a pole at 
location Zi which moves from —1 to +1 as the signal correlation coefficient a moves from 
— 1 to +1 (see Figure 5). For low values of the input signal-to-noise ratio the pole zi 
becomes very nearly equal to a, while for high input signal-to-noise ratio Z\ approaches 
7 and the filter becomes an all-pass filter. 
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Figure 17. Signal Distortion for the HR Wiener Filter as a Function of a for 7 = —0.5 
(Colored Noise) and Different Values of Input Signal-to-Noise Ratio. 

The processing gain (PG) for the optimal filter increases with increasing signal 
correlation and becomes infinity in the limit as o; approaches ±1 (perfectly correlated 
signal). The processing gain decreases as the input signal- to-noise ratio increases indi- 
cating that less improvement is possible when the input signal-to-noise ratio is already 
high. 

The normalized mean-square error (MSE) for the optimal filter decreases with 
increasing correlation and becomes zero in the limit as a approaches ±1. It is fairly flat 
for most values of a and decreases most rapidly as a approaches ±1. The normalized 
mean-square error (MSE) also increases when the input signal-to-noise ratio decreases. 

The signal distortion (SD) becomes zero when 0 = 7, where the filter becomes an 
all-pass filter, and when a approaches ±1, where the signal is completely correlated. 
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It achieves its maximum values for moderately high values of correlation. In the 
case of white noise this occurs around 0.8 < |ad < 0.9. (see Figure 16). 

The performance of the filter for colored noise parallels that of the filter for white 
noise. However, the symmetry with respect to a = 0 is lost. When a = 7, the processing 
gain goes to zero, the mean-square error achieves its maximum value, and the signal 
distortion goes to zero. Maximum values of the signal distortion occur at two distinct 
points above and below the value of the noise correlation parameter 7 (see Figure 17). 
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csr,; 
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III. 



ANALYSIS OF THE FIR WIENER FILTER 



In this chapter we develop expressions for the FIR form of the Wiener filter for 
the case where the signal and the noise are, again, each represented by a first order AR 
model (equations (1-19) and (1.24)). The chapter is divided into two parts. In Part A, we 
derive analytical expressions for the first order FIR Wiener filter, i.e., a FIR filter with 
only two coefficients. In Part B, we use a more general approach to derive expressions 
for a filter of any order. Although Part A treats a restricted particular case, it allows us 
to obtain simple expressions for the measures of performance as functions of the model 
parameters. On the other hand, the results of Part B permit us to plot the measures of 
performance for more general cases, where the FIR filter can be of any order. 

A. THE FIRST ORDER FIR WIENER FILTER 

In Chapter I we introduced the matrix form of the Wiener-Hopf equation (1.16) 
for the FIR Wiener filter and the associated mean-square error (1.17). For the first 
order FIR Wiener filter, the vector of filter coefficients h has only two elements, i.e., 
h = [h{0] h(l)]^. Thus we can obtain expressions for these two coefficients by solving 
equation (1.16) as follows. Since the noise is assumed uncorrelated with the signal, Rx{l) 
and Rsx{l) are given by (1.3) and (1.4). Evaluating (1.20) and (1.25) for lag I = 0 and 1, 
and substituting in (1.3) yields 

-Rx(O) = al + al = aj[l + pin] (III.l) 

and 

^x(l) = + a^j = aj[7 -I- Qpin] (III.2) 

Similarly, from (1.20) and (1.4) we obtain 

RsxiO) = (IIL3) 
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Rs^{l) = acr2 

Using (III.l) through (III. 4) in equation (1.16) results in 



{Pin + 1) 


(apin + 7) 




■ h(0) ■ 


= a^ 


1 


_ {aPin + 7) 


{Pin + 1) 




ft(l) _ 




a 



Then, solving this matrix equation by inverting the matrix, we can express the filter 
coefficients as 



■ h(0) ■ 


pin 


Pin + 1 


— {Oipin + 7) 




1 


. '*(1) . 


{Pin + 1 )^ - [ocPin + lY 


_ -{ocPin + 7) 


Pin + 1 




Oi 



(111.5) 

(111.6) 

The transfer function for the filter is then given by 

H{z) = h{0) + h{l)z~^ 



h{0) = Pin 



{pin + 1 - a^pin - Oil) 
{Pin + 1 )^ “ {o^Pin + lY 



h(l) — Pin 



(a -7) 



{Pin + ly - (apin + lY 



or 

H{z) = h{Qi){ l-z^z-^) 
where the zero of the filter is located at 

Z = Zo = 




(II1.7) 



(m.8) 



or 



Zo = (7-Q) 

° {Pin + 1 - Ot'^Pin - 0 : 7 ) 



(III.9) 



Figures 18 and 19 show the location of the zero Zq of the first order FIR Wiener 
filter as a function of a for different values of 7, and for an input signal-to-noise ratio of 0 
dB and —10 dB, respectively. From these figures we can identify three interesting cases. 
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Figure 18. Location of the Zero of the First Order FIR as a Function of the Signal and 
Noise Correlation Coefficients (o; and 7) for Input Signal-to-Noise Ratio of 0 dB. 



First, for a = ±1 and 7 7^ ±1, and any value of the input signal-to-noise ratio, the filter 
places the zero at zq = Tl- This can be easily verified from equation (III.9). Secondly, 
when Q! = ±1 and 7 = ±1 we see that zq assumes different values. These values can be 
found by setting 7 = ±1 in (III.9) obtaining 

_ ±l-a 

Pin “hi ^ pin T OL 



and evaluating 



Zq — lim 



± 1 - 0 : 



= lim 



-1 



±1 



(III.IO) 



Pin + 1 - Op'Pin T “-^±1 -‘^'OcPin T 1 2pj„ + 1 
Finally, when o: = 7 7^ ±1, the filter places the zero at zq = 0, regardless the value of 
input signal-to-noise ratio. This seems intuitively correct, since when o; = 7 the filter 
cannot distinguish the signal from the noise, and should become an “all pass” filter 
(zq = 0 ). 
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Figure 19. Location of the Zero of the First Order FIR as a Function of the Signal and 
Noise Correlation Coefficients (a and 7 ) for Input Signal-to-Noise Ratio of —10 dB. 

Figure 20 shows the location of the zero of the first order FIR filter for 7 = —0.5, 
and for different values of the input signal-to-noise ratio. The three cases discussed above 
are again evident from this figure. 

1. Processing Gain for First Order Filter 

To obtain an analytical expression for the processing gain defined in (1.29), we 
first find an expression for the output signal-to-noise ratio (1.31) of the first order FIR 
filter. Starting with the general expression [Ref. 5] 

00 00 

^(0= E E Rxil-ki + ko)h{ki)h{ko) 

ko=—oo A:i=— 00 

and assuming that the signal alone is applied to the filter we have 

^.( 0 ) = i2^s{-ki + ko)h{k,)h{ko) 

A:o=0 A:i=0 



38 





Figure 20. Location of the Zero of the First Order FIR Filter as a Function of a for 
7 = —0.5, and for Different Values of the Input Signal-to-Noise Ratio. 






hHl) 



h^O) 



+ 1 



4- 9^(1) J 



and by applying (III. 8) we obtain 

■Ry. (0) = ^^(O)crs(zo - 2zoa + 1) 

By an identical procedure, we obtain 

^.(0) = h^0)cr^(z^ - 2 zo7 + 1) 



(III.11) 



(III.12) 



Finally, by using (III. 11) and (III. 12) in conjunction with (11.11), (1.29) and (1.31), we 
can express the processing gain for the first order FIR filter as 



- (4 + 1 - 2 ^ 00 ^) 

(zo + 1- 2zoj) 



(III.13) 
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Figure 21. Comparison between the Theoretical and Experimental Values of the Process- 
ing Gain for the First Order FIR Filter for 7 = 0 (White Noise) and Input Signal-to-Noise 
Ratio (pin) of 0 dB. 

Figures 21 and 22 show that there is good agreement between the theoretical and 
the experimental values of the processing gain of the first order FIR filter, for the case 
when 7 = 0 and Pin = 0 dB, and when 7 = 0.5 and pi„ = —10 dB, respectively. Figure 
23 shows that the processing gain is independent of the input signal-to-noise ratio at 
three different points. The first point occurs when a = 7, as mentioned before, when the 
filter becomes an “all pass” filter; here the resulting processing gain is 0 dB. The other 
two points occur when a = ±1 and 7 7^ ±1, where zq = ^1, and upon substituting this 
result into (III. 13), we find that the processing gain becomes = 2/(1 ± 7), i.e., 

independent of the input signal-to-noise ratio. 

Figure 23 also shows the general behavior of the processing gain for the first 
order FIR filter in the white noise case, for different values input signal-to-noise ratio. 
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Figure 22. Comparison between the Theoretical and Experimental Values of the Pro- 
cessing Gain for the First Order FIR Filter for 7 = 0.5 (Colored Noise) and Input 
Signal- to- Noise Ratio (pi„) of —10 dB. 



Notice that higher values of the input signal-to-noise ratio results in lower values of the 
processing gain. Intuitively, this happens because when the input signal s{n) is strong 
compared to the noise, the filter cannot make a large improvement. In the limit, as 
Pin — > 00 the processing gain goes to zero (except at o; = drl). 

2. Mean Square Error for First Order Filter 

The (unnormalized) mean-square error can be computed from (1.14) using (III. 3), 
(III.4), (III.5) and (III.6) yielding 

- [a^h(O) -I- oca^sH^)] 



1 



pin 



pin + l- oP'pin - 2q'7 + 

(.Pin 4" 1 )^ (o^Pin "b 7)^ 



(III.14) 
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Figure 23. Processing Gain for the First Order FIR Filter as a Function of a, for 7 = 0 
(White Noise), and Different Input Signal-to-Noise Ratio. 

Alternatively, by applying (III.8), it can be written as a function of Zq: 

_ /i(o)[i _ azo] (III.15) 

Using (III.15) in (1.33) gives the normalized mean-square error as 

MSB = [1 - /i(0)[l - qzq] (III. 16) 

Figures 24 and 25 compare the theoretical and experimental values of the nor- 
malized mean-square error for different values of input signal-to-noise ratio and the noise 
correlation parameter 7. Again, there is a good agreement between the theoretical and 
the experimental curves. Figure 26 shows the behavior of the normalized mean-square 
error for different values of input signal-to-noise ratio. We see that for higher values of 
the input signal-to-noise ratio the mean-square error assumes lower values, as expected. 
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Figure 24. Comparison between the Theoretical and Experimental Values of the Mean- 
Square Error for the First Order FIR Filter for 7 = 0 (White Noise) and Input Signal- 
to-Noise Ratio of 0 dB. 

3. Signal Distortion for First Order Filter 

The signal distortion defined by (1.34) is easily evaluated. It is easy to show that 
the cross correlation between input and output of a linear system is related to the input 
autocorrelation function by the convolution relation [Ref. 5] 

00 

Rxy{l)= E h{k)R^{l-k) (III.17) 

k——oo 

Hence, for our first order filter we have 

^...( 0 ) = 'Zh{k)Rs{-k) 

k=Q 

= <7s(/i(0) + ah(l)) 

= a'^h{0){l — azo) (III.18) 
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Figure 25. Comparison between the Theoretical and Experimental Values of Mean- 
Square Error for the First Order FIR Filter as Function of a for 7 = 0.5 (Colored Noise) 
and Input Signal-to-Noise Ratio of —10 dB. 



Then, by applying (III. 18) and (III. 11) to (1.34), we obtain 

(III. 

Figure 27 shows the theoretical and experimental values of the signal distortion 
for different values of a, and for 7 = 0 (white noise) at a input signal-to-noise ratio of 0 
dB. We again observe good agreement between the experimental and theoretical results. 
As in the case of the HR filter, the distortion is zero for a = 7 and a = ±1, and achieves 
a maximum value at a point 0 < < 1. In Figures 28 and 29, the noise parameter 

7 has fixed values of 0 (white noise) and 0.5, respectively, and the signal distortion is 
plotted for different values of the input signal-to-noise ratio. Notice that for lower values 
of input signal-to-noise ratio the signal distortion assumes higher values. In addition. 
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Figure 26. Mean-Square Error for the First Order FIR Filter as a Function of a for 
7 = 0.5 (Colored Noise) for Different Values of Input Signal-to-Noise Ratio. 



notice that, regardless of the input signal-to-noise ratio, for a = 7 or for a = ± 1 , the 
signal distortion is zero. This can also be easily verified from (III. 19). 



B. PERFORMANCE MEASURES FOR HIGHER ORDER 
FIR WIENER FILTERS 

For FIR filters of higher than first order (P > 2), it becomes extremely difficult 
or impossible to derive analytical expressions for the filter coefficients and zeros of the 
filter in term of the basic parameters a, 7 , and pin- However it is still useful to be 
able to compute theoretical values for the processing gain, mean-square error, and signal 
distortion in these cases. Therefore, in this section we develop a different approach to 
derive expressions for these performance measures, which can be plotted as a function 
of the various parameters. With these results, we can compare the performance the FIR 
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Figure 27. Comparison between the Theoretical and Experimental Values of Signal Dis- 
tortion for Different Values of a and for 7 = 0 (White Noise) at a Input Signal-to-Noise 
Ratio of 0 dB. 



filter of any order to that of the optimal IIR Wiener filter when they are applied to the 
first order signal and noise models of equations (1.19) and (1.24). 



1. Processing Gain for Higher Order FIR Filter 

The output power of a filter expressed in the form (1.15) can be written as 

Rs{0) - E{t/^(n)} 

= E{(h^x) (x^h)} 

= h^E{xx^}h 

= h^R^h (III. 20) 

Therefore, the output power when the signal and noise act alone is given by 

Rj/,(0) = h^R,h (III.21) 
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Figure 28. Signal Distortion for the First Order FIR Filter as a Function of a for 7 = 0 
and Different Values of the Input Signal-to-Noise Ratio 



and 

Ry^{0) =h'^R^h (III.22) 



where and R;, are the signal and the noise correlation matrices, which have the form 
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(III.23) 



Thus from (III.21), (III. 22), (1.29) and (1.31) we obtain 



h^R,h 1 

h^R,,hpi„ 



(III.24) 
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Figure 29. Signal Distortion for the First Order FIR Filter as Function of a for 7 = 0.5 
and Different Values of the Input Signal-to-Noise Ratio 

The processing gain produced by the Wiener filter- can be obtained by substituting the 
solution to the Wiener-Hopf equation (1.16) into (III. 24). Specifically, from (1.16) and 
(1.18) we have 

h = (R, + R^)-'f, (III.25) 

Since we have all the necessary expressions, we can plot the values assumed by the 
processing gain (III.24) for different values of a, 7, and pin- 

Figure 30 shows the behavior of the processing gain as a function of a for 7 = 0 
(white noise) and for different values of input signal-to-noise ratio. Notice that the 
processing gain has different maxima for a = ±1, depending on the order of the filter. 
These maximum values can be easily predicted for the white noise case (7 = 0), as 
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follows. For Q; = 1 and 7 = 0 the signal and noise correlation matrices (III. 23) become 
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R.= 






•• ^5 
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,2 ... 


0 
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0 ••• 





(III.26) 



Under these conditions (III. 25) shows that h(0) = h{l) = ... = h{P — 1). By applying 
these results to (III.21) and (III. 22), we obtain 



^ 2 ( 0 ) 



and 

h^'R^h = aj P ^2(0) 

Finally, substituting these expressions into (III.24) results in 



PG = P 



A similar analysis shows that PG = P also for a = —1. 

This result shows that unlike the HR filter, the processing gain for an FIR has 
a finite bound, which is equal to the filter length. This result can be easily checked in 
Figure 30.^ Notice also that in general, for any value of a, larger length filters produce 
increased processing gain. 

2. Mean Square Error for Higher Order FIR Filter 

The normalized mean-square error (1.33) for any FIR filter is not difficult to obtain. 
Using (1.15) and (I.l) we can write 

E |(s(n) - y(n))^| = E {(s(n) — s — f))(s(n) - h - h)} 

= cr2 - 2 f , + R, h -h h’’ R„ h 

^Note that the vertical scale in Figure 30 is in dB 
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Figure 30. Processing Gain for the FIR Filter of Length P for 7 = 0 (White Noise) and 
Input Signal-to-Noise Ratio of 0 dB 



where the cross terms disapear because the signal and noise are uncorrelated. Then, 
substituting this expression in (1.33) we obtain 

2 h"’ f , - h"' R, h - h"" R„ h 



MSB = 1 - 



(III.27) 



The result (III. 27) applies to any FIR filter. For the Wiener filter we can simplify 
the expression by noting from (1.16) and (1.18) that 



(Rs + R;,)h - = 0 

Therefore, for the Wiener filter (111.27) simplifies to 

1 'j' 

MSB = 1 ^ (111.28) 

This result can also be obtained directly using (1.17). 
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Again, we have all the necessary elements to plot the normalized mean-square 
error as a function of the model parameters and the input signal-to-noise ratio. Figure 
31 shows the normalized mean-square error plotted as a function of a, for 7 = 0 and input 
signal-to-noise ratio of 0 dB. It can be seen that higher order filters result in lower values 
of mean-square error, £is expected. However, for the longer filters, there is significant 
improvement in performance only for very high values of the signal correlation parameter 
a, i.e., the ability to reduce mean-square error depends heavily on the correlation present 
in the signal. 




Figure 31. Mean-Square Error for the FIR Filter of Length P for 7 = 0 (White Noise) 
and Input Signal-to-Noise Ratio of 0 dB 

3. Signal Distortion for Higher Order FIR Filter 

To derive an expression for the signal distortion for the FIR filter of order P, we 
write the response of the filter for the signal alone using (1.15) as ys(n) == h’" s. Thus, 
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we have 



E {s{n)ys{n)} = E {s(n) §} = 



(III.29) 



and 



E {y,(n)2} =E{h^s h} = R, h (III.30) 

Substituting these expressions into equation (1.34) then produces the desired expression 

(h’’ is? 



SD = 1 - 



Rs h a2 



(III.31) 



This can be evaluated for the Wiener filter by substituting the solution (III.25) for h. 

Figure 32 shows the behavior of the signal distortion for the FIR filter of different 
orders P, for 7 = 0 and input signal-to-noise ratio of 0 dB. The behavior of this measure 
is similar to its behavior in the case of the HR filter. The signal distortion is 0 dB 
for a = ±1 and for a = 7 and achieves a maximum value around a value ± 0 . 8 , 
depending on the length of the filter. In addition, the value of the signal distortion is 
greater for larger filter lengths. 



C. SUMMARY 

The analysis carried out in this chapter for the FIR Wiener filter shows that this 
filter has performance qualitatively similar to that of the HR filter, although for the case 
of a first order AR signal and noise model the performance, in terms of the mean-square 
error and processing gain, is never quite as good. 

For the first order FIR filter, which has a single zero, the location of the zero 
changes from +1 to —1 as the signal correlation coefficient increases from —1 to + 1 . 
When the signal and noise have the same correlation coefficient (a = 7 ) the zero moves 
to the origin (resultind in an all-pass filter); when both signal and noise are perfectly 
correlated (a = 7 = ± 1 ) the zero moves to a location that depends on the input signal- 
to-noise ratio (see (III. 10)). This results in a steady state estimate for the signal which 
is equal to a constant fraction of the input observation sequence. 
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Figure 32. Signal Distortion for the FIR Filter of Length P for 7 = 0 (White Noise) and 
Input Signal-to-Noise Ratio of 0 dB 

The performance measures for the FIR filter approaches those of the HR filter as 
the order of the filter increases. Figures 30, 31, and 32 show plots of the three performance 
measures for the FIR Wiener filter of orders 2 , 4, and 10. These performances can be 
compared to those for the HR Wiener filter in Figures 33, 34, and 35. A significant 
difference between the two types of filters is that the processing gain of the FIR filter, 
when the input signal is perfectly correlated is not infinite, but is limited to a value P 
(the filter length). Also the mean-square error does not go to zero when a = ±1 as in 
the IIR filter case. However, the signal distortion for the FIR filter does go to zero for 
a: = 7 and o; = ± 1 , as in the IIR filter, and it is not as large as for the IIR filter. 
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Figure 33. Processing Gain for the IIR Filter for 7 = 0 (White Noise) and Input Signal- 
to-Noise Ratio of 0 dB 
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Figure 34. Normalized Mean-Square Error for the HR Filter for 7 = 0 (White Noise) 
and Input Signal-to-Noise Ratio of 0 dB 
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Figure 35. Signal Distortion for the IIR Filter for 7 = 0 (White Noise) and Input Signal- 
to-Noise Ratio of 0 dB 
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IV. 



THE EXTENDED OPTIMAL FILTER 



We can conclude from the results of the previous chapters that the Wiener filter, 
although optimal in the mean-square error sense, is not necessarily optimal with regard 
to minimizing the signal distortion. This chapter addresses the possibility to improve 
the signal distortion performance of the filter, at the expense of increasing the output 
residual noise. In other words, we may be able to improve the signal intelligibility with 
some slight increase in mean-square error due to noise. 

Ephraim and Van Trees [Ref 7] have suggested a more general filter for speech 
enhancement, based on one measure of the signal distortion; here we have adapted this 
idea using the signal distortion measure defined in Chapter I. This chapter formulates 
this more general problem and derives the solution using an FIR filter. It then provides 
an analysis of the performance of the generalized FIR filter for the first order AR signal 
and noise model, and finally suggests how the generalized filter may be used to obtain 
an appropriate trade-off between signal distortion and mean-square error. 

A. THE EXTENDED OPTIMAL FILTER PROBLEM 

We begin with the filter depicted in Figure 3, and the responses ys(n) and t/,,(n) 
due to the filter acting on the signal and noise alone. The problem as suggested in [Ref 
7] is to minimize the distortion while constraining the residual noise power to satisfy 

E{vl{n)}<a^ (IV.l) 

where is some chosen level of residual noise power. In this chapter, we apply this 
procedure using the signal distortion measure defined in (1.34). In the context of an FIR 
filter, the problem can be formulated as to minimize (III. 31) subject to the constraint 

E{s,»}=h’'R,h<<r; (IV.2) 
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Let us first consider the procedure for minimizing the signal distortion without 

the constraint (IV.2). To do this it is necessary to maximize the term 

(h^ r,)" _ f,rfh 
h^’R^h h^Rsh 

appearing in (III. 31 ). This can be done by constraining the denominator to be any 
constant 

h^Rjh = const 

and maximizing the numerator. Thus, by forming the Lagrangian [Ref. 5 , 8] 

£ = h^fjf j h + /i(const — h^Rjh) 

where is a Lagrange multiplier, and setting the gradient of the Lagrangian to zero we 
obtain 

V/j£ = 2fsfjh — 2/iRsh = 0 

This shows that h must satisfy 

f, f J h = /I R, h (IV. 4 ) 

i.e., h is a generalized eigenvector for the two matrices fj f J and Rj. The only non- trivial 
solution to (IV. 4 ) is given by 

h = RJ^ fj 

which is equal to the all-pass filter h = [1 0 0 ... 0]^. 

When we introduce the constraint (IV.2) we need to form a new Lagrangian 

£ = -(h^fs)f Jh + /ii(const - h^Rjli) + /i2(o’r — Rt? h) (IV. 5 ) 

where fxi and /i2 are Lagrange multipliers. Setting the gradient to zero then yields 

V/i£ = 2 fj f J - 2 /ii Rj h - 2 jLt2 R77 h = 0 




or 

fs fj h = + AR,,)h 
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where A = (jl^/ H\- In this case, we see that the solution is given by 



h = (R5 + AR,) 



(IV.6) 



with jUi = f J (R5+ AR^) 5. The parameter A can be related to through the constraint 

(IV. 1). In particular, substituting (IV.6) into (IV.l) with the equality yields 

f J (R, + AR^)-' R^ (R, + AR^)-' f, = 



or 

tr f J R,, (R5 + AR„)“^ = cr^ 

The solution (IV.6) for the extended optimal filter is the same as that of a Wiener 
filter with modified noise power. Some particular special cases are of interest. For 
A = 0 we obtain the all-pass, h = [1 0. . .0]^ (as we have already noted), resulting in 
the maximum residual noise power and zero signal distortion. For A = 1 we 

have the Wiener filter which minimizes the residual noise power = cj^. By choosing 
an appropriate value 0 < A < 1 we obtain minimum distortion with residual noise 

In summary, by choosing a value for A we can gain more control over the signal 
distortion at the expense of increased residual noise'and mean-square error. In the studies 
presented below, it is shown that this added flexibility has some definite advantages for 
noise filtering. 



B. PERFORMANCE OF THE GENERALIZED FILTER 

By using equation (IV.6) in (III. 24) we can study the processing gain of the filter 
as a function of the parameter A and the signal correlation coefficient a. Figure 36 shows 
the results of this computation for 7 = 0 (white noise) and an input signal-to-noise ratio 
of 0 dB. Notice that the processing gain is lower for lower values of A, demonstrating 
that there is less noise removal for A < 1. 

We can also substitute (IV.6) into equation (III. 27) to obtain an expression for 
the mean-square error as a function of a and A. Figure 37 shows these results for the 
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Figure 36. Processing Gain for the Generalized Wiener Filter as Function of a and A for 
-y = 0 (White Noise) and Input Signal-to-Noise Ratio of 0 dB. 

same case as Figure 36, i.e., white noise and an input signal-to-noise ratio of 0 dB. Again, 
there is decrease in performance for values of A < 1, going from minimum mean-square 
error for A = 1, which corresponds to the optimal Wiener filter, to maximum for A = 0, 
which corresponds to an all-pass filter, that does not remove any noise from the observed 
sequence. 

Finally, we can apply (IV.6) to (III.31) and plot signal distortion as a function 
of a and A. Figure 38 shows the results for the same case as before. Here we see that 
the signal distortion of the filter improves for higher values of A, going from maximum 
distortion for A = 1 (Wiener filter) to minimum distortion for A = 0, where the resulting 
all-pass filter does not alter the signal. 

The rate of change of the measures of performance PG, MSB and SD as a func- 
tion of A allows one to make useful trade-offs. Notice from Figures 36 through 38 that 



60 




Figure 37. Mean-Square Error for the Generalized Wiener Filter as Function of a and A 
for 7 = 0 (White Noise) and Input Signal-to-Noise Ratio of 0 dB 

for values of A greater than approximately 0.4 the performance of the filter in terms of 
processing gain and mean-square error does not change significantly, while the s signal 
distortion decreases significantly over this range of values of A. This implies that the gen- 
eralized filter with an appropriately chosen value of A may be more desirable in practical 
applications involving noise removal than the standard Wiener filter. Since the solution 
(IV. 6) for the filter is equivalent to that of a Wiener filter with an addition parameter, it 
is convenient to adapt this filter for use in a short-time noise removal algorithm. We will 
see in the following chapter a practical application of the new algorithm, which provides 
good performance and more control over the results 
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Figure 38. Signal Distortion for the Generalized Wiener Filter as Function of a and A 
for 7 = 0 (White Noise) and Input Signal- to-Noise Ratio of 0 dB. 
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V. APPLYING EXTENDED OPTIMAL 
FILTERING TO UNDERWATER SIGNALS 



A. INTRODUCTION 

In this chapter we apply the solution of the extended optimal filter expressed 
in (IV. 6) to the short-time filtering algorithm developed by Frack in [Ref. 4]. This 
algorithm assumes that both the signal and the noise correlation functions (Rs{l) and 
Rij{l)) are unknown a ‘priori] therefore they must be estimated from the data at hand. 
Since only x{n) — s{n)+rj{n) is observed, however, only Rx{l) can be estimated directly. 
Nevertheless the nature of the problem provides a way to compute the needed statistics. 
Since the signal is very short (on the order of seconds or milliseconds) compared to the 
time over which the noise statistics are likely to change, an estimate for Rq{l) can be 
made from the received data prior to the onset of s(n). By using (1.3), this estimate can 
in theory be subtracted from Rx{l) to produce an estimate of Rs{l). 

Although there is no explicit analytic expression for the parameter A, we can 
obtain the value of A corresponding to any desired residual noise level by solving (A). In 
the case of white noise, this is equivalent to writing (IV.2) as 

II h |p= <7^ (V.l) 

and then ( by IV.6) solving for A from 

II (R. + AR,)-'f, ||^= ^ (V.2) 

Therefore, by setting the level of the residual noise cr^ that can be tolerated, we can 
assign values for A in each segment by iterating on (V.2) until the equality is satisfied, 
thus obtaining the value of A that minimizes the signal distortion. Thus, in principle, all 
of the quantities to perform the extended optimal filtering over a short time interval can 
be computed, and this can be repeated over successive blocks of data. 

The estimate of Rs{l) by subtraction of the estimated correlation functions is 
not well formed because there is no guarantee that such an estimate will be positive 
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y[n] = s(n) 




Figure 39. Prewhitening in Short-Time Extended Filtering Algorithm. 

(semi)definite. This problem is mitigated if the noise is white because the procedure 
then involves subtraction of only a single parameter, the white noise variance, from the 
estimated at lag zero. Further, the estimate of this single parameter has lower variance 
than the estimate of the correlation function as a whole. Therefore before any further 
steps, the entire data set is processed by a linear predictive filter that whitens the noise. 
After noise removal, the data is processed by the inverse filter as shown in Figure 39. 

The prewhitened data is segmented into blocks where an estimate of the local cor- 
relation function Rx{l) is formed for each segment. Also, for each segment, the algorithm 
finds the value of A, since A is a function of the Rs{l),Rr,{l), and o^. Extended optimal 
filtering is then performed for each segment using a filter designed for the segment, and 
the data is processed by the inverse filter to undo the effects of the prewhitening. In 
performing the filtering, the data is processed both forward and backward through the 
extended optimal filter. (This gives an approximation to a symmetric noncausal filter of 
twice the length.) Since the optimal filters are different for each block, discontinuities at 
the boundaries can arise. The effect of such discontinuities can be minimized by using 
points from the adjacent segment to filter the early points of the current segment. In the 
algorithm, the data is actually processed twice. The data is first segmented and filtered 
and the resulting frames are weighted by a triangular window (see Figure 40). The data 
is then resegmented using frames shifted by half of the frame length, filtered again and 
weighted by a triangular window. The two weighted sets of data are then added to pro- 
duce the final result and minimize any effects that may occur at the boundaries between 
frames. Additional details of this processing can be found in [Ref. 4]. 
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B. RESULTS 

To obtain a quantitative measure of the signal distortion, we first need to have 
a clean signal s(n), and obtain the response of the filter to this signal s{n), as 

explained in Chapter I. In this way we can demonstrate our conclusions about the im- 
provement in the signal distortion accomplished by the extended filtering algorithm. 

For these experiments, an observation sequence x{n) was generated by adding a 
white noise sequence with variance < 7 ^ to the clean signal s(n). The sequence x{n) was 
then processed using the algorithm described above to obtain the filter parameters for 
each segment of this sequence. These same filter parameters were then used to process 
the corresponding segments of the clean signal s(n) to obtain ys{n). 

We can now compare the effects of the filter on the signal with respect to signal 
distortion. In order to do that, we introduce the estimates 



^ , (En S W ys{n)f 


(V.3) 


MSE = 7 

En 


(V.4) 



where the summation is over all points in all segments. 

For all the following experiments, the segment (frame) length is 100 points, the 
filter order is 35, and the AR model order for the noise was arbitrarily set to 10 (this 
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parameter could be set to any value since the noise is white). The first 1,000 points data, 
consisting of noise, was used for the noise sequence. 

Figure 41 shows the result of filtering a clean synthetically generated short pulse 
signal with 1,400 points (41(a)), contaminated by white noise with variance = 0.36 
(41(b)), using the traditional Wiener filter. The result, shown in Figure 41(c), indicates 
that the filter is effective in removing the noise. The signal distortion estimate (V.3) 
for this case is SD = 0.0358, and the normalized mean-square error estimate (V.4) is 
MSE = 0.2249. 

Now, by applying the extended filtering algorithm for the case where more resid- 
ual noise can be tolerated, we can obtain lower signal distortion. Figure 42 shows the 
graphical results for this case where the residual noise power is set to correspond to 30% 
of the input noise power (cr^/cr^ = 0.30). Part (d) of the figure shows the value of A 
that resulted for cr^/cr^ = 0.30. Notice that A assumes values less than 1 for some of the 
segments. The signal distortion estimate for this case becomes SD = 0.0541, which is 
an improvement of about 3%, and MSE = 0.1910, a degradation of about 0.7%, when 
compared to the traditional Wiener filtering shown above. 

Figure 43 shows the results for an extreme case when the residual noise power is 
70% of the input noise power. Here we can see that A assumes very low values for most 
of the segments. The signal distortion estimation for this case is very low SD = .0358, 
decreasing the signal distortion by a factor of 55%, and MSE = 0.2249, a degradation of 
about 18%. 

The following table summarizes the results of these experiments using three dif- 
ferent settings for the ratio cr^/cr^. The values in parenthesis indicate the increase or 
decrease from the values achieved by the Wiener filter. 

The results shown here demonstrate that when more residual noise can be toler- 
ated, the signal distortion produced by the filter decreases significantly while the mean- 
square error increases by only a modest amount. 

The noise removal algorithm was also applied to real underwater acoustic data. 
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estimates 


Wiener 

Filter 


Generalized Filter 


ailal = 0.3 


<y'iK = 0.5 


= 0.7 




0.0556 


0.0541 

(-3%) 


0.0466 

(-19%) 


0.0358 

(-55%) 


MSE 


0.1896 


0.1910 

(+0.7%) 


0.1985 

(+5%) 


0.2249 

(+18%) 



Table I. Results from the Generalized Filter Experiments. 



Figures 44 and 45 shows the results of applying the extended filtering technique to data 
representing a killer whale song. In Figure 44 we used the traditional Wiener filtering 
process, while in Figure 45 we allowed the residual noise to be 20% of the input noise 
power. For this case, it is difficult to quantify the performance, since the true underlying 
signal is unknown. However, upon listening to the signal, the results are at least com- 
parable to those of the Wiener filter and the slight increase in residual noise is the price 
paid for lower signal distortion. 
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(c) 



Figure 41. Results of the Application of the Extended Optimal Filtering Technique to 
a Synthetically Generated Short Pulse Signal with Added Low Power White Noise, for 
Minimum Residual Noise (A = 1). (a) Original Clean Data, (b) Original Data plus 
White Noise, (c) Processed Data. 



68 




(c) 




(d) 



Figure 42. Results of the Application of the Extended Optimal Filtering Technique to 
a Synthetically Generated Short Pulse Signal with Added Low Power White Noise, for 
Residual Noise Power Equivalent to 30% of the Input Noise Power, (a) Original Clean 
Data, (b) Original Data plus White Noise, (c) Processed Data (d) Values of A for Each 
Segment. 
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Figure 43. Results of the Application of the Extended Optimal Filtering Technique to 
a Synthetically Generated Short Pulse Signal with Added Low Power White Noise, for 
Residual Noise Power Equivalent to 70% of the Input Noise Power, (a) Original Clean 
Data, (b) Original Data plus White Noise, (c) Processed Data (d) Values of A for Each 
Segment. 
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(b) 



Figure 44. Results of the Application of the Extended Optimal Filtering Technique to a 
Killer Whale Song for Minimum Residual Noise (A = 1). (a) Original Noisy Data, (b) 
Processed Data. 
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(b) 



Figure 45. Results of the Application of the Extended Optimal Filtering Technique to a 
Killer Whale Song for Residual Noise Power Equivalent to 20% of the Input Noise Power, 
(a) Original Noisy Data, (b) Processed Data. 
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VI. CONCLUSIONS 



A. THESIS SUMMARY 

The first three chapters of this thesis provide an extensive analysis of the HR 
and FIR Wiener filters for removing additive noise from a desired signal. Both signal 
and noise are represented by first order autoregressive (AR) models. Three performance 
measures, namely processing gain, normalized mean-square error, and signal distortion 
are defined and evaluated for the Wiener filters based on these models. We believe that 
the behavior of these filters in terms of the defined performance measures is typical even 
for other cases with more general signal and noise models. 

It was seen that the FIR filter has performance similar to that of the IIR filter, 
although it does not have the same ideal limiting characteristics such as infinite process- 
ing gain when the signal becomes perfectly correlated. We also showed that while both 
forms of the filter provide lower mean-square error and higher processing gain as the 
signal becomes more correlated, signal distortion is not a monotonic function of correla- 
tion. In particular the signal distortion for both forms of the filter peaks in a region of 
moderately high correlation and increases when the input signal-to-noise ratio decreases. 
This distortion introduced by the filter may be undesirable in some applications. 

Chapters IV and V deal with a way to improve the signal distortion resulting 
from the Wiener filter by considering a more general filtering problem. The result is 
a generalized solution that can reduce the signal distortion of the Wiener filter, with 
some increase in the mean-square error and residual noise. This solution was adopted 
in a short-time filtering algorithm developed by Frack [Ref. 4] and tested. The results 
illustrate that the extended filter provides more flexibility and allows more control over 
the signal distortion introduced by the filtering process. 
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B. SUGGESTIONS FOR FUTURE RESEARCH 

The results from this thesis can drive some further discussions, including the 
performance of aural tests to evaluate the performance of the extended optimal filtering 
technique when applied to real sonar data, and application to real-time signal processing. 

In the time set aside to perform this thesis research the major portion of the effort 
involved the analysis of the Wiener filter and its generalization. Relatively little time 
was left for considering implementation details and more extensive testing on real data. 
For more efficient implementation, the nonlinear equations that specify the parameter A 
in terms of the residual noise need to be inverted more efficiently. This could be done 
by a more effective iterative method or by inverting the function at a number of points 
and performing table look-ups. 

While the new algorithm is known to permit a trade-off in signal distortion versus 
mean-square error, the practical effects of this trade-off have not been truly evaluated. 
For example, formal listening tests have not been conducted to see (over some class 
of signals) if listeners preferred the results of the generalized filter over those of the 
conventional Wiener filter or, indeed, if they could hear the difference. The greatest 
benefits may come in the use of this new algorithm in conjunction with further signal 
processing steps which may be sensitive to any distortion of the signal introduced by the 
noise removal process. These are all topics for possible future study. 
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